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(54) PICTURE PROCESSOR 
(57)Abstract: 

PROBLEM TO BE SOLVED: To keep contrast even when a 
picture is displayed in a display system with a narrow 
density contrast by processing a group of pictures 
consisting of plural pictures, executing gradation correction 
by each picture in the group of the pictures and compositing 
each gradation-corrected picture to prepare one picture 
with wide dynamic range. 

SOLUTION: A gradation correction circuit detects a 
luminance signal from divided picture information dd, ee 
outputted from a divided area picture information extracting 
circuit and detects whether a pixel constitutes an edge or 
not concerning each pixel in a picture from a luminance 
signal mm outputted from a Y-signal detecting circuit 51. 
Based on the edge detecting signal nn of an edge detecting 
circuit 52 and the signal mm, an edge histogram showing 
appearing frequency to a luminance level is calculated. This 
gradation correction makes it possible to compose pictures 
in the neighborhood of the center of the density range of a 
wide dynamic range picture to carry out to generate by 
picture composing processing circuit later. 
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/C'SX':;i;^m[D]g§5 2 C(D:r,yiy^m^^5 2(0 

lHlSS5 3i. C<7:)x^;':;^tX h^^Aft»liI3S5 33&>6 

F§iaiiIE^#tt<h J^cC h > :^ - ^ PgillffliEl^ 

h->:^-::^^IilK5 4<b. C<Dh- 
>:^7-:/S:^|pI885 4 5??^6Hi:t?$n^ h->;^-:/#14 

ft-^p P t±mn^m^mmt±iE»mw$,\mic c <t 

CC«-:5liT±iB^fiIiij#tf fSd d . e e CDPgPffliE^tf 

c^fflmo^fOiffll^ltfR f f . g g ^m;'7-r ^P^MiE 
[0062] 3^*j. ±iax t?«itB[ass5 2 m« 

So be 1^<D— j9SWJ^cfx^^5^^Hi:^*-^b-^CcJ:0:7 

(>cj:^^m^M(Dmm (mtf±iaA/D^g5<o 

-jlyiy^^&r^ti^X. fEGj iU. ^ 

^-e3^cCt^n«X';.v^ff«^ roj <b-r^2fii1f«^Hi:/7 

[0 06 3] c'^i^xmQ^mi^fc^^tsimmcj^^m 

ST-Sci^S^-c^^o c<7:)<fc^^cPgill?BiEt3:. \kxm9i 

^ c i -c ^ J; ^ ^ -r -2) ffl IE <b o r c ^ ^ o 
[0 064] iS/c, 07»±fBx-^t?t;^ h^^Att^ 
[pl58 5 3 OCfcC^r. |^SX':;5^t:X h^-^A^ft^-r^ 

[0 0 6 5] ST. 07 (A) \it. ^^UfcVf^^m^l^ 
[0066] -7^. mi (B)W. **SP»^*-C^<b L 

r iiasp^c f^:^^ -oxmch i^^t^M^m^ mx^ 

m^itij^>h'r^<Df>cMLx. ^^m^<Dmm^^:r. 

^yi^x^^m^(>a^mmm^m^^:X6 AtLxijo> h 
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[0 06 7 ] rj::^. cn6S7 (A) . 07 (B) ©fS] 

[0 0 6 9 ] 08 (A) mwi^<Dmmmmff^m^ 

[0 07 0 ] ^CD*)^i:bf5W0it^^[^CC^S^S 

[0 07 1 ] c(Dt^ic\t. ^ti'etio:>mmc:^ii^n 

gft-^cDtxh^^A^i, mt^ccoas (A) (fCmir 
<fc5K:^j:D. (LE- I mag e) «Jt?5 20 

(SE - Image) iitbfW^^C^T^^Cfi-^ri^^iC 

[0 07 2 ] ^cr\ ±ia06(C^L/cJ:^i^j:F§iStSiE 
liIiB39. 40^cfclir. ^LJ:'?tir^a:^^i-^ 

V \y>v^mm<oiRm\y>t^<D^^^ (ymax/z) 

[0 0 7 3 ] 08 (B) ti. ±ia08 (A) <0^^i^^ 
[0 07 4] C^bX. mhtlfc^-(i^^ y ^\y>i><0 

tpx^<(D\mmm^mmr^cti)^x^. jg^wtc. 

[0 07 5] ;X(C. 0 9 . ±iaa#^^i3;S[sISS4 1 

[0 0 7 6 ] C<D«!!iS?J^^'MltS$n'5<b. ^T. ±iaP^ii 40 
ffliEliISS3 9. 4 0;&i6ffl:tJ5n^P§ia*ijE3ti/c^>S« 
laiRtffllSegSE (x. y) . SegLE (x, y) 

ttfRS eg (x. y) ^m^i^ts C^'ryy'SA 1) o 

[0 07 7 ] ^com. Wi^'<i*^ -J ^ v^^ymmm,^ 

^iW-r6«i|8WDR (x. y) (DVmit^^y-yX (X-r 
3) o 

[0 07 8] ^ur. ^nwmns e g < l . j ) 

LEDa t SiXhhtP^'Stf^^Wfii^ {:^f'vy'SA 50 
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4) . LEDa t ar^aig^tCti. &r . g. bic^ 
UTSegLECi. j ) ©r-^^±iB«I^WDR 

(i. n ^^m-r^ (;^f--^r^s4 9) « 

[0 07 9]—:^. ±faxf*7 :7*s 4 4^c*5Cir. le 

Da t ar^^cCC^lS'&k:^. ^X^C. ^fUM^tt^S e g 
(i. j ) :05SEDa t ar*a;5^53i?>^*iJ»TL (Xf* 
•:.:;^S4 5). SEDa t a-C^>^^-&«:«. S-r. 
g. b^^Ur S e g SE ( i . j ) cDf^-^^±fa«l 
i^WDR(i. j ) tCtSj^-r^ (Xt-'v-^'S 5 0) o 
[0 08 0] ^/c. ±fB:^f*^»:/S4 5tcfcCir. SE 
Da t a'Ci^cCl^ii'^^5^. SegLE(i. j ) CD^fe 
JiK^j'. SegLEr, SegLEg. SegLEbOrt 
(OmX^OS^^mi^X (Xf-*:^7'S4 6) . ^XCCgLSSt 
R^^^iirMfO. ^CDSL^RfecfcOXScD:?^^^ 
CCjSCr. 3lJRfKFt4«iST€:0^/c»UC^^T^ 
f'*y::''S4 7) o 

[0 08 1 ] CC^C. CCD5MJR#ttfiiSTCD^:^Si 

\ XDa t ai^ct:)f#^f6H (04 tc*5t:f SKfil 
Th l^JcfcO'iafiiTh 2<7)fiH) CCfcl^r. CS;&^<i:(D 
egc7);^^^-C^-S;&>^7S-rMiaCs t (CS) ^UkA 
ICDcfc^^c^ftT^o 
[0 08 2] 
[^1] 



[0 08 3] CCDg8tfi:Cs t (CS) {H. (Th 
1 ^CS<Th 2) ]H^^^>XOt^h ICC^iSifJnT^ 

/c2i?)tc. c S(Di^^ § «:^*Dii-c^JS!liii«W$R ( S e g 
LE(i. j).SegSE(i,j)) ^mRT^n 

[0 0 8 41 ^xcc. umL^wMhfctc^x. ±Mm 

mCsi (CS) ^SL^RO^AfilRMAX^^D^C 

tx. cs(Di^^$=^^m.orj:f)iib. ^^^^(o^mm^ 
mn^miR^^-^^comm^ ^ > ^ ak:?? ^ c tf)^x^ 

[0 08 5] 
[IK2 ] 

lOtherwize 



ST 



[0 08 6] CCDl^^2t?«, ms^M^mCS(0:k^ 

tj-^^ MISCs t (CS) *i/jN$t,i (#iBL,ruS^SS 

ili^tffgS e g LE ( i . j ) KML.-caS50'Og-^l.^ 

l^Cs t (CS) *S7^tt,i (#Murc»-5^f!na^ifl8 
S e g L E ( i . j ) {C0gb-cafRa=©S^t,»35iJ:blSS*) 
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iexf"';;:7'S4 9Mf or. . g, b^cKUrSe 
gLE(i. j ) COx-^*Jiifi^i§SWDR (i. j) 

MfoT. SegSE (i. j ) (Z^T^-tS^^rJJB^illcW 

DR ( i , j ) ic^mr^o 

[0 08 8 ] ±3SLfci:^3^j:Xr'>:/S4 4*»6Xy*^i 10 
v'S 5 0 (DjftS^cj: 0 . iii^^)S*^LS[5IK4 1 J^^ffl^-^ 

[0 0 8 9 ] ±ia>^7" :;'*S 4 9 ^/c&^;^f' :7'S 5 0 
>>^®^tflRWDR (x. y) ^ai::^;LT (X-f'-iFT'S 

5 2) . i^T-r^o 

[0 0 9 0] c<Dcl:^tcfem^«A«i©:^#^*#iitL 20 

x^mmi^inm^miR-r^ c <b j: o . :55^f«SRii{tt$Bj^5 

MI XDa t ar$>^fli|lc«. x ^ IfMS^iSU/cttSg 

[009 1 ] tj:^. ±mxummmm^mmtmmmm 
[ 0 0 9 2 ] */c. c cx^ttmi^tim^m^n'f-t;^ ^ 

[0 0 9 3 ] c<DJ:^rj:mi O^Jgff^^&c 1^- 

[0 09 4 ] ^fc. ^^^mjE^^o^mi^omiEmMMm 

^^f&'T -S C i <J: D . 0f C ^gP^3^>^ 6 ^ I ^ ^ rjA 

ssffltcs ^ r n > h ^ X h :«>^'ffif$ s titc^mxjK ^-i-r 

[0 0 9 5 ] $^^C. ^^Lci:^ (b'r€>Ji;^V:^^ ^ 
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[0 09 6 ]-eLr. P^iUffliElHlK^^'l^r. 

[0 09 7] */c. CCDt;^ hi:/^Ak:S-^i^TP^P!ffl 
X^6. Stor. JNFSi^^cDifi^SB^«:B8^^:3>h^ 

[0098] mr. iiifi^*or)®^fii:acciSGrt:>^ h 
A ^^^t* ^ b ^ -if ^ cfc ^ CC 0 /c 

mmjE^n ^ct tu^. 
[0 09 9] Jttc. mmMm^^mi(fC^i^>x. mmm 
^(D^mi^i^-r^ir^xmi^m^^mf&r ^m^<D^m^(o 

u-<)vtmstr -s c <h k: cfc 0 . miEn^tmm ( 
mytmm^^ts) t:^^jEmytmm<D^M^n-^x. ± 
x(D&m^<oiy-<M^f>mjEx^^im^M}Em^mmt 

'ox^n^ti^mjEm^mM(on&^\^±'r^ct^^x 

[0 1 0 0 ] 01 Ot^fbmi 5»*ffe?90ll2<^)^^ 
m^w^Ufci><DX$>^. C(D02CD^JfeffJ.|gK:*5l^r. 

on^^ftvxmm^^^o. ^tLxm^^j^.^tc-oi^ 
xcD^mM-^^o 

[0101] ccomzcommBmomj-tf^'ycDmm 
±Mo tern 1 cD^i^jfjsi(D0 1 (tc^fkL fc^<ot 

[0102] :^XCC, m 1 0 «:t7^ -^M-^iftJillffiCD^^ 

[0103] c<ommm^(ot;j>^'7m^mmm(fCidi.^ 

e2ircxt}$n^^^(fCtj:-,x\.^6. 

[0 104] cn6CDY/C^g|[pIS86 1. 6 2^^^. 
Xf)m^m^<DT. g. bfm (R. G. B) {cS-:Jli 
)l¥gfi#Y (SI 2 (A) #M) <!:feMm#Cb. 

c r i^c^s;3ic^-r<fc^^c^gf-r€>o 

[0105] 

[^3]Y= 0. 29900R+0. 58700G 

+ 0, 1 440 OB 

Cb=-0. 1 687 4R-0. 33 1 2 6G+0. 5 
OOOOB 

Cr= 0. 5000 0R-0. 4 1 8 6 9G-0. 0 
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8 1 3 1 B 

[ 0 1 0 6 ] bx^m^titcm<Dmmrsm%sE(^c 
(tm^mmmnmm - l e ijmmwmm^mfc^^r. y i> 

(01 2 (B) #M) o ^ci:*5. CCrtJ. -r 
(?^a'rs^l 2 (C) #M) i3?^Hi;t?$n€»Ci^Aj: lo 

[ 0 1 0 7 ] C ^ UT^tH^n/cX i^tf fgfg-^n n - 
SE, nn-LE(b. ±iaY/Ci$>gtlnI^6 1, 6 2:^^ 
6©iSSM^mm-SE. mm~ L E*5<tO'fe^ft^q 
q-SE. qq-LEi&CS-^l^r. Hi^ffliE^Kfc^ 
PiiaffljE|plK6 5.6 6 r«^B#raS:)^6S E &C^^ilj0 

[ 0 1 0 8 ] 0 1 1 CCDFglHffliElHlSSB 5 , 6 6 CD 

E^C0^€>BI@ffliE[Bl8S6 6 (iC-D{,>X<DmM^n^o 
[0109] F§ia?iiEliIK6 6 CC*5t>r53:. jWJK^-^k: 
a-:5< X e>tf ?g^-^n n - L E ;&^#S5[ii£^ t h 

nr. X5;i^tSffift#nn-LEOU--<;l/&c*t-r5Ui^ 

[0 110] -Jiaw^j:e^ill^^c4ba^rts. ±iBx-:.t^ 
^^nrfcO. m«S12 (C) (fC7jkrJ;:^rj::r.yty 
[0111] C^L/rX":.^:^tX hy-^A{^fi£§P7 ir 

gifiii'reciCC^fc'^. ± 2 a02O(Di3ffl^^^^n 
So 40 
[0 112] CCDggfilStfflgPT 2>&>6ffl;^$nSI3ffii 
±iH)^ffifi^Ox y ly\mm^ n n - L E . *5J: ?>1SiS 
fi-^mm-LE(C«':5l^r. ^fmHif^jliR^S/cS^g 

fli+2 act0 4>Atl^X';;i;^^:S^^/cl^gg{ii-2 aJ:0 

So 

[0113] COJ:^icx-:;>^fcx h^-^ACD^igeM 
■r S c i < i;?nS^<D;5^^Jflm-r S so 
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[ 0 1 1 4 ] c ^ urtttb^n/cW^Ai^cCx t^^^osf 

h^yJ^i'^JSSS^l AX{t. m«012 (D) CC^-TJ: 
[0 115} C<D«SStX hi5^^Af^^«»7 4rf^^$ 

ft^^«*rS/cfi^)CC. ±fe)S¥gt;^h^7AJSr;^7'^->r 

[0 1 1 6] -o^D. ±^u/cmio^Jfejf^.^tcfci>r 
^ ft S J: ^ U /c (D r ^ S o 

[0117] /c/cL. ^CDfi]g<D:tf':;5^r>t^<Db>^ h 
[0 118] ccx{t. n^^:^ V^'yh.^iO^t^y:^ 

aSiJ^cSfcX h:J/^A^®ffiKCCf#S^S^fflt.^S. 
[0 119] rf^^%. Sfgfc:;^ h^'^A^O (L) . 

cn^*r'5^'r>jKK:^u/cfcxhi/^A^T (D 
<h-rsi. o (L) 3{>^6T (L) tc^^-rs/c&osst 

©:S;4k:^-r<fc^cc;rj:So 
[0 12 0] 
[|»4] 

^2 



r(£)=o(i)* 



at'' 



[0121] cccc, la^ J »:3 ^>3> 
k \x i'cDfi^^ r i7 »^ i^r >ttcc^^*r S 

fc:>?.h^^AO (L) C0^ffimoi-2fg|iS^fflU 

So 

[0122] C^Ur. CCD«r«. 012 (E) 

[0 1 2 3] c(7)<fc^K:. A^^nsa^ftcc. a«<!: 

J^j:SbX h^^^A^3ifflW^C:i)STS/ci?>tC. ^mtm 
rn^n 1/ r *>^#:tc*f S c <b :*^r # S » 
[ 0 1 2 4 ] C ^ br^6n/cS^bX h ^'^AiilB 
Jfjg b X h ^^'7 A i ci: «3 . PgPIMiE#tt^^^K/c S 
^fftfflM^fi£a57 etC^oC^r. JlSbX h^-^A?:gS 
bXht5^7AtC^tfeTS|JgiS^«lft*a^01 2 (F) CC 

[ 0 1 2 5 ] ^or. }Limmm^^ui it^ho^ 
mkconmnnt^mm q q - l e <h i^m-^^ . mm. 
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[0126] 

[»5 ]R = Y+ 1.40200Cr 
G = Y-0. 344 1 4Cb-0. 71417Gr 
B = Y+1.7720 0Cb 

[0127] ±lB»Sfi-^af^gP7 ii)^h<o^ 

im.(Dm&m^<Dm.-xm ( l e - ym a x ) *5ct z^^m^ 

fit ( L E - YM I N ) (O^m^. «^S/hffl«ltaa57 8 10 

^c^oi^rtfci. -ecD^^^JiS:* im i otc4dCr:f ^ r r - l 

r r - S Er*>^o ) *±fBiii«^RS5Q;SleIK6 7 

[0128] ^X^C, S 1 3*5cfcO'a 1 A\t±Mmw.^^ 

-So tii6. cti601 3fcJ:afai 4t^. Hi^^^ffi 
{H^ 6 7 iBigcr)tP^±2 -oOSS 

[0 12 9] C<D«Q!S*5ggj^$n-&i. ±Mmm 20 

»iE[iIS§6 5. 6 6;^»e>ta;^7$n^P^i)i?iIE^ti/cSiii 
«1f$SSE (x. y) . LE (x. y) 4dJ:0'±iH)liS 

E-YMAX. SE-YMIN). (LE-YMAX. 
LE-YMIN) ^K^iitf {XTv':fS^ 1) « 

[0130] ^of^. ^u^-i-r % ui^tymmm^ 
^mr^mimDR (x. y) ^^z^^^m^w-mimm 

^mmSA (x. y) OWMit^n^^^ (XT-yy'SQ 30 
2). LEiiil^tt$B^^-r^ (;^7^';.:7-S 6 3) o 
[0 1 3 1 ] -eUT. ^^Wb/cr. g. bCCl^^LE 
r (i. j) . LEg (i. j) . LEb (i. j ) Cc 

7^-:;:7'S6 4) , 

[|S6] LEv=0. 3LEr(i,.1)+0. 59LEq(i, 
D+O. 1 1 LEb(1,.1) 

[0 13 2]-eL/r. CCDLE yj&iLE- YMAXWT 
J&^oSE-YMI NJ:9*i/h3l^>5PS3^«:«»r 

u (:^f^yy'seb) . )R:&(D^ff^;&Wc$nTii'2)it 40 

Sr. g. bOcMLrLE ( i . j)<^r--:§? 
*±fEMi^WDR ( i . j ) itC^mr^ C^K^ vV'S 6 
7) , 

[0 13 3] JifS;^-r^/7'S6 Stcfci^r. 

LEy:05LE-YMAXJ:«:i4>A^< . ;0^oSE-YM 
I NWir^^T!^^^?^^^^^^]^^ (>5.7"':;-7*S6 6) . M 

LrSE(i. j ) CD'r-^^±iB^i|8WDR (i. 

j ) cc^mr^ c^T- :7's 6 8 ) o 50 
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[ 0 1 3 4 ] cn^XT- ^ :7*S 6 STt^^ex-r •:;:7'S 6 8 

[0 13 5] tfc. ±fBXf'«:.:7*S6 6K:4dtir. ::4>^j: 

g. b^MbTLEd. j ) (D-r-^^^iSE ( i , 
j ) C3Df^-^<h<D^i&fii«:±fB^«WDR ( i . j ) ^ 

^;&^S?&^:S:$iJ^-r^¥m{bMilEt»$StS A ( i . j ) CC 1 

[0136]^ LX. mm^(D^i^WT hfciP^iP' 
ilBXf*^:^ :7*S 6 3 CCMor^XCDH^O^^^f 
M^tff$B^^*r^ {Xt^vV'SI 1 ) « 

[ 0 1 3 7 ] ^ ur . w-mimmmnsk ( i . j ) 

1 t?^^:0^5;0^^fiJWtr (Xr'^:^r?'S7 2) . 1 
^li^tCJi. WDR ( i . j ) ^cfi^Cv^: U/cmK5 x 

icfcc^r. ^iii«^^«iSlPlS86 7)&s¥?t<b*IS:<bor 

[0138] —75". ±fBXf^-:^7'S7 2CC*5C>r. ¥it 

{k^.i^tffRSA ( i . j ) i)^\ xf^\.>tmmtrifcm^ 
[0139] -eur. ^ffli^ccotir(3!)^s*mTU/c 

5!pS5^^$«BrUr (Xf*^:/S7 4) . J^TUTC^fel^ 
it-^tCti. ±fBXf*-:':?*S 7 1 cc^-^rso'^^tf 

tffgWDR (x. y) ^^tii^X {:^r'v-:fS7 5) . 

[0 140] C(DJ:^?^j:*i!tS^tf ^Ci«:J:0. mi 5 
tc^-r<t^^. lii^JSgfii:65SE-YMI NiLE-Y 
MAXtc$l^^n/c$I« mi 5 (A) (D J: ^ &cM:^cr>ia 

T^iSB^raS^tLECDf^-dfiJaBSPaSifeS EOf^-^S? 
6(D^^i^m^mi>^mn(OM^ (M i x t u r e > ^tf 
^CtttCfjiK}. SE-YMI NiLE-YMAX<D{5in 

(LE - i ma g e) ^ffll^ SE-YMIN 
i L E - YMA XCD(pin;0>ilfe[75^igl^:&<fc 0 4>0J'5>l^i® 
m(rC-:>l^X\itm^fSM%<D-(jt-l^ (SE- imag 

e) ^m^^^Ct(^tj:^. 

[0141] C<om^(fat. LE-YMI NcfcOfeHil^ 

mm(fC-^^^x{t&mmmyc(D>( (LE-imag 

' e) ^mi^>^Ct(tCrj:K}. SE--YMAX<:fcD4>9^^li 
iiiSK:oiir«^B$raS3fe©'Y^-i^ (SE-imag 
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[ 0 1 4 2 ] c ^ ur, m^o^mjEm^m^^mL-c 
[ 0 1 4 3 ] a 1 0 mi^^f&(imMKd 

[0145] ^fc. m.^^±% vip\y>^j^m9i^^'r 
flic^sft»;&^ hm^^^u^'t xmmm^m,-:^ r > h ^ ;^ 

[0 146] ^hiiC. mmmiEm^iiC^y^^x. 

[0147] ^Ur, ^tmij^CDtX h^^A^^;SU 

[0148] fiD^r . mi^^f&!iBtmMm^c:hi>x . m~ 
mmiiCrx^^xmrnmiE^tifcmi^m^tm^^t^m 

uns^i^ ^ tifcmmi^^mit^mr c i tc j: «9 . 

[0 149] ni^-^^siPiK^fet^-c. mmm 

]E^n-otc^mi^^mmiiLmm(^cm^xmmm^<D^jE 

^^mmoxj:^mjEm^^ci5:\,>m^<D^^m^^i' ^ 
v{;^\y>zymmcmm-r:hcti)^x^^. 

[0150] 5 6^. iil^^fi£«!!;ailKtC*Jlir. MJE 50 
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m^mm^^f&LxjK^^a-^ y^v>iymm^^f&ir 

^ C i J: *3 . *cDSia^d:fff grffliE 

4^Pfi^^flI*^J■r C t i)^X^ 
[0151] :^mm±Ml^fcmSmi^^^cm'&^ 

[0152] 

WiCctih^^^i^x—com^A-rx V ^ vzy'jm^^ 

[0 1 5 3 ] ^/c. ff5jt^2CC^^:*:|feBJCDili^M®^ 

mffcn^fcm(fCctih^^fS.Lx~<Da:^^i^ ^ 
n^ctf)^x^^. 

[0154] $ e)^c. imms (iCj:^:^mM<Dmi^mm 

f&^n"? c tf>^x^ ^j:^i.cmmm]E^n^fcii>(iC. m 
miy>i^^w^('cmi^^x^mimwit'r^ctf>^'^mt 

[0155] n^mAiiCj^i^^mmmmmmWiCj:. 

[0156] Mmm^(tc^^^mmmi^mmmm(rc^ 
[0157] mym6(fCj^^:^^m(ommmmm(fC^ 
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[0158] Mmmi (^x^^mMommmmmmcj^ 

[0159] m>^ms(^<k^'^mm<Dmi^mmm^(^j;: 
fete. H^'&fiE^^tcfci^r. mmm]E^n^fc^mi^ 

mmr^cti^^x^^. 30 

[0160] i»5}<ii9 cCcl:^*^?BcDii«iOjil^gocJ: 

[0 16 1] if ^311 o(fCj:^:$:^m<Dmmmmms(fC 
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[Sffi<D®Wj:i^Hj] 

1 ] :^mB<Dm 1 CDllififf?SIC9m^:^^^<DS2|s:W 

[0 2] ±iam 1 cD||j5S^J^.Cc4t5C:t^:t^^ ^(t-^^3;SliI 

[03 ] ±iB^ 1 (ommimoij^ ^ym^Witmmmc^ 
[04 ] ±iB^ 1 <Dmtmm.<ommm^%mmc^i^ 
[05] ±ie^ 1 (omimm(D6^m%mmmmtb]tim 

[06] ±iaill(D*«gff$SSCCfct'f-SPgiB?SiEl9l88<D« 
[07 ] ±iB0 1 CD*]teJ&SIC3[>X t;^t: X h Aftgt 

T0O 
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(54) PICTURE PROCESSOR 

(57)Abstract: 

PROBLEM TO BE SOLVED: To keep contrast even 
when a picture is displayed in a display system with a 
narrow density contrast by processing a group of 
pictures consisting of plural pictures, executing 
gradation correction by each picture in the group of 
the pictures and compositing each gradation- 
corrected picture to prepare one picture with wide 
dynamic range. 

SOLUTION: A gradation correction circuit detects a 
luminance signal from divided picture information dd. 
ee outputted from a divided area picture information 
extracting circuit and detects whether a pixel 
constitutes an edge or not concerning each pixel in a 
picture from a luminance signal mm outputted from a 
Y-signal detecting circuit 51. Based on the edge 
detecting signal nn of an edge detecting circuit 52 and 
the signal mm, an edge histogram showing appearing 
frequency to a luminance level is calculated. This 
gradation correction makes it possible to compose 

pictures in the neighborhood of the center of the density range of a wide dynamic range 
picture to carry out to generate by picture composing processing circuit later. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



[Claim(s)] 

[Claim 1] The image processing system characterized by to provide an image amendment 
means are the image processing system which processes the image group which consists of 
two or more images picturized on different exposure conditions to the same photographic 
subject, and generates the extensive dynamic range image of 1 . and perform gradation 
amendment for every image in the above-mentioned image group, and an image composition 
means compound each image by which gradation amendment was carried out with this image 
amendment means, and generate the extensive dynamic range image of 1 . 
[Claim 2] It is the image processing system which processes the image group which consists 
of two or more images picturized on different exposure conditions to the same photographic 
subject, and generates the extensive dynamic range image of 1 . A division means to divide 
into a proper exposure field and an unsuitable forward exposure field based on predetermined 
picture signal level about each image in the above-mentioned image group, An image 
amendment means to perform gradation amendment of the proper exposure field divided by 
this division means for every above-mentioned image. The image processing system 
characterized by providing an image composition means to compound the proper exposure 
field for every image by which gradation amendment was carried out with this image 
amendment means, and to generate the extensive dynamic range image of 1 . 
[Claim 3] The above-mentioned image amendment means is an image processing system 
according to claim 1 or 2 characterized by being what amends the gradation of each above- 
mentioned image so that image composition can be performed near the center section of the 
concentration range of the extensive dynamic range image which it is going to generate with 
the above-mentioned image composition means. 

[Claim 4] A feature-extraction means by which the above-mentioned image amendment 
means extracts the description component from this picture signal by filtering for every image 
in the above-mentioned image group. A histogram generation means to generate a histogram 
from the near pixel of the description component extracted by this feature-extraction means, 
A gradation amendment property generation means to generate a gradation amendment 
property based on the histogram generated by this histogram generation means. The image 
processing system according to claim 1 or 2 characterized by coming to have a gradation 
amendment means to perform gradation amendment of this picture signal using the gradation 
amendment property generated by this gradation amendment property generation means. 
[Claim 5] The above-mentioned histogram generation means is an image processing system 
according to claim 4 characterized by being that to which weighting at the time of generating 
a histogram is changed according to the pixel location in an image. 

[Claim 6] A feature-extraction means by which the above-mentioned image amendment 
means extracts the description component from this picture signal by filtering for every image 
in the above-mentioned image group, A description component histogram generation means to 
generate the histogram about the description component extracted by this feature-extraction 
means, A threshold calculation means to compute the threshold for choosing an object pixel 
from the distribution condition of the description component histogram generated by this 
description component histogram generation means. An object pixel selection means to 
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choose an object pixel from this picture signal based on the threshold computed by this 
threshold calculation means, A histogram generation means to generate a histogram from the 
object pixel chosen by this object pixel selection means, A distribution modeling means to 
model the histogram of the object pixel generated by this histogram generation means in 
predetermined distribution, A gradation amendment property generation means to generate a 
gradation amendment property based on the histogram of the object pixel generated by the 
above-mentioned histogram generation means, and the histogram modeled by the above- 
mentioned distribution modeling means, The image processing system according to claim 1 or 
2 characterized by coming to have a gradation amendment means to perform gradation 
amendment of this picture signal using the gradation amendment property generated by this 
gradation amendment property generation means. 

[Claim 7] The above-mentioned image composition means is an image processing system 
according to claim 1 or 2 characterized by coming to have an averaging means to calculate 
such averaging, and a smoothing means to perform data smoothing to the pixel processed by 
this averaging means when two or more picture signals by which gradation amendment was 
carried out with the above-mentioned image amendment means exist about the same pixel 
location. 

[Claim 8] The above-mentioned image composition means is an image processing system 
according to claim 1 characterized by coming to have a proper exposure pixel selection means 
to choose the pixel near the proper exposure in the pixel group of the same pixel location for 
every pixel location as a pixel which constitutes an extensive dynamic range image to each 
image in the above-mentioned image group by which gradation amendment was carried out 
with the above-mentioned image amendment means. 

[Claim 9] In case the above-mentioned image composition means compounds two or more 
proper exposure fields and generates the extensive dynamic range image of 1 When the lack 
field which turns into an unsuitable forward exposure field in each images of all in the above- 
mentioned image group exists By using the applicable field of the maximum exposure image in 
the above-mentioned image group using the applicable field of the minimum exposure image in 
the above-mentioned image group, when this lack field is an exposure undershirt on the other 
hand when this lack field is exposure over The image processing system according to claim 1 
or 2 characterized by coming to have a field adjustment means to fill up the above-mentioned 
lack field. 

[Claim 10] A maximum chrominance-signal detection means to detect the maximum of two or 
more chrominance signals with which the above-mentioned division means constitutes this 
picture signal about each image in the above-mentioned image group, The image processing 
system according to claim 2 characterized by coming to have a maximum chrominance-signal 
comparison means to enable judgment of the above-mentioned proper exposure field and an 
unsuitable forward exposure field by comparing with the above-mentioned predetermined 
picture signal level the maximum in the chrominance signal detected by this maximum 
chrominance-signal detection means. 
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DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to an image processing system and the image 
processing system which generates the extensive dynamic range image of 1 from two or more 
images picturized on different exposure conditions in more detail. 
[0002] 

[Description of the Prior Art] Various things were proposed from before, and the image 
processing system which compounds two or more images picturized on different exposure 
conditions, and generates the large image of a dynamic range corresponded by compressing a 
dynamic range, when concentration range, such as a monitor and a printer, displayed the large 
image of the dynamic range compounded with such an image processing system by the 
comparatively narrow display system. 

[0003] as for compression of this dynamic range, it is common to be performed by the 
property (a logarithm — for it to be similar to the property) according to the usual gamma 
characteristics, and it has become the thing it was made not to, spoil the contrast of main 
photographic subjects or a background if possible. 

[0004] As an example of such a technique, it compresses into JP.5-314253.A in the property 
which is proportional to the logarithm of brightness about a high brightness region, and what is 
compressed in the property which was proportional mostly at brightness is indicated about 
the low brightness region. 

[0005] The example of the compression means of such a conventional dynamic range is 
explained with reference to drawing 16 . 

[0006] For example, based on the short-time exposure signal SIGSE of the light exposure 
ratio 1 :8, and the long duration exposure signal SIGLE, an extensive dynamic range image is 
created and the case where it is compressed is explained. 

[0007] In this case, if the amount of incident light to which the output level of the long 
duration exposure signal SIGLE reaches saturation (100%) is set to alpha, since the case 
where a light exposure ratio is set to 1:8 is taken for the example, the amount of incident light 
to which the output level of the short-time exposure signal SIGSE reaches saturation is set 
to Salpha. 

[0008] First, the extensive dynamic range signal SIGWDR is created by doubling the output 
level of the short-time exposure signal SIGSE eight, and compounding it with the above- 
mentioned long duration exposure signal SIGLE further. 

[0009] next, in order to double this extensive dynamic range signal SIGWDR with a narrow 
monitor, a narrow printer, etc. of a concentration range, an output level is proportional to the 
amount of incident light in a small part — as — a part with a large output level — the amount 
of incident light — receiving — a logarithm — he compresses an output level to become-like 
and is trying to acquire the signal SIGLST finally amended 
[0010] 

[Problem(s) to be Solved by the Invention] However, in the compression property by Prior-art 
means which was mentioned above, although the contrast of an extensive dynamic range 
image is mostly maintained at the condition at the time of photography about a low brightness 
region, since the variation of an output level to the variation of the amount of incident light 
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becomes small so that it becomes a high brightness region, contrast will fall. 
[001 1] Since a person will exist in a high brightness region comparatively in a scene when the 
person who does not restrict that main photographic subjects not necessarily exist in a low 
brightness region in actual photography, for example, is in a dark scene is photoed using a 
stroboscope When the above-mentioned conventional compression is applied, a person's 
contrast will be spoiled and it will become the image of the smooth sensibility called so-called 
sic***** image. 

[0012] Moreover, in the Prior art mentioned above, since the same compression property is 
always taken, without taking into consideration the description of images, such as an edge and 
luminance distribution, depending on an image, the extensive dynamic range image which is 
not so good-looking may be generated. 

[0013] This invention is made in view of the above-mentioned situation, and even if it displays 
on the narrow display system of a concentration range, it aims at offering the image 
processing system which can generate the extensive dynamic range image which can hold 
contrast from a low brightness region to a high brightness region. 
[0014] 

[Means for Solving the Problem] In order to attain the above-mentioned purpose, the image 
processing system by the 1st invention An image amendment means to be the image 
processing system which processes the image group which consists of two or more images 
picturized on different exposure conditions to the same photographic subject, and generates 
the extensive dynamic range image of 1, and to perform gradation amendment for every image 
in the above-mentioned image group, It has an image composition means to compound each 
image by which gradation amendment was carried out with this image amendment means, and 
to generate the extensive dynamic range image of 1 . 

[0015] Moreover, it is the image processing system which the image processing system by the 
2nd invention processes the image group which consists of two or more images picturized on 
different exposure conditions to the same photographic subject, and generates the extensive 
dynamic range image of 1. A division means to divide into a proper exposure field and an 
unsuitable forward exposure field based on predetermined picture signal level about each 
image in the above-mentioned image group, It has an image amendment means to perform 
gradation amendment of the proper exposure field divided by this division means for every 
above-mentioned image, and an image composition means to compound the proper exposure 
field for every image by which gradation amendment was carried out with this image 
amendment means, and to generate the extensive dynamic range image of 1. 
[0016] Furthermore, in the image processing system by the 1st or 2nd above-mentioned 
invention, the image processing system by the 3rd invention amends the gradation of each 
above-mentioned image so that image composition can be performed near the center section 
of the concentration range of the extensive dynamic range image which the above-mentioned 
image amendment means tends to generate with the above-mentioned image composition 
means. 

[0017] In the image processing system according [ the image processing system by the 4th 
invention ] to the 1st or 2nd above-mentioned invention A feature-extraction means by which 
the above-mentioned image amendment means extracts the description component from this 
picture signal by filtering for every image in the above-mentioned image group, A histogram 
generation means to generate a histogram from the near pixel of the description component 
extracted by this feature-extraction means, It comes to have a gradation amendment 
property generation means to generate a gradation amendment property based on the 
histogram generated by this histogram generation means, and a gradation amendment means 
to perform gradation amendment of this picture signal using the gradation amendment 
property generated by this gradation amendment property generation means. 
[0018] The image processing system by the 5th invention changes weighting at the time of 
the above-mentioned histogram generation means generating a histogram in the image 
processing system by the 4th above-mentioned invention according to the pixel location in an 
image. 
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[0019] In the image processing system according [ the image processing system by the 6th 
invention ] to the 1st or 2nd above-mentioned invention A feature-extraction means by which 
the above-mentioned image amendment means extracts the description component from this 
picture signal by filtering for every image in the above-mentioned image group, A description 
component histogram generation means to generate the histogram about the description 
component extracted by this feature-extraction means. A threshold calculation means to 
compute the threshold for choosing an object pixel from the distribution condition of the 
description component histogram generated by this description component histogram 
generation means. An object pixel selection means to choose an object pixel from this picture 
signal based on the threshold computed by this threshold calculation means, A histogram 
generation means to generate a histogram from the object pixel chosen by this object pixel 
selection means, A distribution modeling means to model the histogram of the object pixel 
generated by this histogram generation means in predetermined distribution, A gradation 
amendment property generation means to generate a gradation amendment property based on 
the histogram of the object pixel generated by the above-mentioned histogram generation 
means, and the histogram modeled by the above-mentioned distribution modeling means. It 
comes to have a gradation amendment means to perform gradation amendment of this picture 
signal using the gradation amendment property generated by this gradation amendment 
property generation means. 

[0020] The image processing system by the 7th invention comes to have an averaging means 
to calculate such averaging when two or more picture signals by which gradation amendment 
of the above-mentioned image composition means was carried out with the above-mentioned 
image amendment means exist about the same pixel location, and a smoothing means to 
perform data smoothing to the pixel processed by this averaging means in the image 
processing system by the 1st or 2nd above-mentioned invention. 

[0021] The image processing system by the 8th invention comes to have a proper exposure 
pixel selection means to by.which the above-mentioned image composition means chooses 
the pixel near the proper exposure in the pixel group of the same pixel location for every pixel 
location as a pixel which constitutes an extensive dynamic range image to each image in the 
above-mentioned image group by which gradation amendment was carried out with the 
above-mentioned image amendment means in the image processing system by the 1 st above- 
mentioned invention. 

[0022] In the image processing system according [ the image processing system by the 9th 
invention ] to the 1 st or 2nd above-mentioned invention In case the above-mentioned image 
composition means compounds two or more proper exposure fields and generates the 
extensive dynamic range image of 1 When the lack field which turns into an unsuitable forward 
exposure field in each images of all in the above-mentioned image group exists When this lack 
field is exposure over, it comes to have a field adjustment means to fill up the above- 
mentioned lack field by using the applicable field of the maximum exposure image in the 
above-mentioned image group using the applicable field of the minimum exposure image in the 
above-mentioned image group, when this lack field is an exposure undershirt on the other 
hand. 

[0023] In the image processing system according [ the image processing system by the 10th 
invention ] to the 2nd above-mentioned invention A maximum chrominance-signal detection 
means to detect the maximum of two or more chrominance signals with which the above- 
mentioned division means constitutes this picture signal about each image in the above- 
mentioned image group, It comes to have a maximum chrominance-signal comparison means 
to enable judgment of the above-mentioned proper exposure field and an unsuitable forward 
exposure field by comparing with the above-mentioned predetermined picture signal level the 
maximum in the chrominance signal detected by. this maximum chrominance-signal detection 
means. 
[0024] 

[Embodiment of the Invention] Hereafter, the gestalt of operation of this invention is explained 
with reference to a drawing. Drawing 9 shows the 1st operation gestalt of this invention from 
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drawing 1 , and drawing 1 is the block diagram showing the fundamental configuration of an 
electronic camera. 

[0025] The image sensor 1 for this electronic camera becoming in the color CCD of the 
veneer type which has electronic shutter ability etc., carrying out photo electric conversion of 
the photographic subject image, and outputting as a picture signal, The lens 2 for carrying out 
image formation of the photographic subject image on this image sensor 1 , and drawing and 
the shutter style 3 for controlling the passage range and pass time of the flux of light which 
passed this lens 2, The amplifier 4 which amplifies the picture signal with which removal of a 
noise component was performed in the correlation duplex sampling circuit which is not 
illustrated after being outputted from the above-mentioned image sensor 1, A/D converter 5 
for changing into a digital signal the analog signal amplified with this amplifier 4, The image 
processing system slack camera digital disposal circuit 6 which performs various kinds of 
processings which are later mentioned to the signal digitized by this A/D converter 5, AF for 
detecting AF (automatic focus) information, AE (auto exposure) information, and AWB 
(automatic white balance) information in response to the digital output from above-mentioned 
A/D converter 5, AE. and the AWB detector circuit 7, The compression circuit 9 which carries 
out compression processing of the image data from the above-mentioned camera digital 
disposal circuit 6 (JPEG), Memory card I/F14 which performs control for recording on the 
memory card 15 which mentions later the image data compressed by this compression circuit 
9, The memory card 15 which records image data by control of this memory card I/F14, 
DRAM1 1 used as working-level month memory in case color processing of image data etc. is 
performed. The memory controller 10 which controls this DRAM11. and PCI/F16 which is an 
interface for transmitting the image data currently recorded on the above-mentioned memory 
card 15 to (personal computer PC) 17 grade, Reproduce and display the display circuit 12 
which controls LCD13 mentioned later, and the image data recorded on the above-mentioned 
memory card 15 by control of this display circuit 12, or LCD 13 which displays various kinds of 
photography conditions concerning this electronic camera etc., the stroboscope 19 which 
emits light in the illumination light for illuminating a photographic subject, and the timing 
generator (TG) which generates the timing pulse for driving the above CCD 1 — 18 — The 
input key 20 which comes to have a trigger switch for carrying out the directions input of the 
switch and photography actuation for setting up various kinds of photography modes etc.. It 
connects with the above-mentioned camera digital disposal circuit 6, the compression circuit 
9, the memory controller 10, a display circuit 12, memory card I/F14, and PCI/F16 through 
the bus line. While receiving Above AF and AE, the input by the detection result and the 
above-mentioned input key 20 of the AWB detector circuit 7, or the luminescence information 
by the above-mentioned stroboscope 1 9 It has the above-mentioned lens 2, a diaphragm and 
a shutter style 3, a timing generator 18, the camera digital disposal circuit 6, a stroboscope 
19, and CPUS that controls this whole electronic camera including each circuit connected to 
the input key 20 or the above-mentioned bus line, and is constituted. 

[0026] The usual photography mode which photos one image and makes it image data in this 
electronic camera. The extensive dynamic range photography mode in which approach in time, 
perform photography of two or more images with which exposure differs, compound these 
images, and the extensive dynamic range image of 1 is obtained. Choose in hand control by 
actuation of the above-mentioned Input key 20, or detect a white jump of the picture signal 
from the above-mentioned image sensor 1, and it makes whether CPUS judges automatically 
and is made to choose photography mode. According to the selected photography mode, 
photography actuation is controlled by CPUS. 

[0027] namely, when photography mode is usually chosen When the picture signal for one 
screen is acquired from the above-mentioned image sensor 1 by one photography and 
extensive dynamic range photography mode is chosen by photography actuation on the other 
hand The picture signal for two or more screens (for example, two screens) from which the 
light exposure from an image sensor 1 differs by the photography to one photographic subject 
is acquired with the well-known means according to combination with a diaphragm and the 
shutter style 3 as the electronic shutter ability or this electronic shutter ability of an image 
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sensor 1. In the above-mentioned camera digital disposal circuit 6, the image data according 
to photography mode is processed. 

[0028] Next, drawin g 2 is the block diagram showing the configuration of the above-mentioned 
camera digital disposal circuit 6. The configuration shown in this drawin g 2 shows the circuit 
for compounding the image of 1 based on the image by the short-time exposure SE. and the 
image by the prolonged exposure LE as an example, and serves as an example supposing the 
case where perform short-time exposure SE first and prolonged exposure LE is performed 
after that, as photography sequence further. 

[0029] The camera signal-processing process circuit 31 which this camera digital disposal 
circuit 6 performs various processings to the video signal aa outputted from above-mentioned 
A/D converter 5 in response to the AWB information outputted from the above CPU 8, and 
generates a picture signal. The switch 32 switched based on the signal jj from CPUS which 
shows whether photography mode is usually photography mode about the output destination 
change of the picture signal bb outputted. or it is in extensive dynamic range photography 
mode from this camera signal-processing process circuit 31, When switched to an extensive 
dynamic range photography mode side by this switch 32 The switch 34 whose picture signal of 
the furthermore switches the output destination change according to whether it is a thing 
concerning the short-time exposure SE. or it is a thing concerning the long duration exposure 
LE, The switch control circuit 35 which controls switch actuation of this switch 34 based on 
the image change-over control signal kk from the above CPU 8, The image data buffer 36 
which accumulates the image data concerning the short-time exposure SE for the one screen 
when the output from the above-mentioned switch 34 is a thing (bb-SE) concerning the 
short-time exposure SE, The division means slack image field dividing network 37 which 
calculates image field division data so that it may mention later from these data when the 
output from the above-mentioned switch 34 is a thing (bb-LE) concerning the long duration 
exposure LE, The image data concerning the short-time exposure SE read from the above- 
mentioned image data buffer 36, The image data which starts the long duration exposure LE 
from the above-mentioned switch 34 is inputted. The isomerism rate means slack division field 
image information extract circuit 38 which divides each image data based on the division field 
information cc calculated by the above-mentioned image field dividing network 37, and 
extracts division image information, The image amendment means slack gradation amendment 
circuit 39 which carries out gradation amendment of the division image information dd 
concerning the short-time exposure SE divided by this division field image information extract 
circuit 38. The said image amendment means slack gradation amendment circuit 40 which 
carries out gradation amendment of the division image information ee concerning the long 
duration exposure LE divided by the above-mentioned division field image information extract 
circuit 38, The division image information ff concerning the short-time exposure SE after the 
gradation amendment outputted from the above-mentioned gradation amendment circuit 39 
and the division image information gg concerning the long duration exposure LE after the 
gradation amendment outputted from the above-mentioned gradation amendment circuit 40 
are compounded. The image composition means slack image composition processing circuit 41 
which generates the extensive dynamic range image of 1, By a force terminal being connected 
to this image composition processing circuit 41 much more, and other input terminals being 
connected to the above-mentioned switch 32, and interlocking and being switched to this 
switch 32 based on the signal jj from the above CPU 8 Usually, it has the switch 33 which 
outputs the signal hh which shows the image information in the picture signal in photography 
mode, or extensive dynamic range photography mode as an output signal ii, and is constituted. 

[0030] Drawing 3 is drawing showing the situation of the processing signal in the camera 
digital disposal circuit 6 at the time of extensive dynamic range photography mode. 
[0031] For example, the case where the person who is a backlight under the bright 
background is photoed is considered. 

[0032] First, although a photograph will be taken by exposure comparatively proper about a 
bright background if short-time exposure SE is performed, an image which a person becomes 
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dark and was crushed black is obtained (refer to bb-SE). 

[0033] On the other hand, although it will white-fly about a bright background if long duration 
exposure LE is performed, about a person, it becomes comparatively proper exposure (refer 
to bb-LE). 

[0034] Since it is the photography of these bb-SE and bb~LE which made the person the 
subject on the other hand at least here, the division field information cc for dividing into the 
field used as the proper exposure in this long duration exposure image and the field used as 
unsuitable exposure with reference to bb-LE is created in the above-mentioned image field 
dividing network 37. 

[0035] This division field information cc includes the information which distinguishes whether 
that pixel is suitable exposure, it is unsuitable exposure, or they are these middle exposure 
about each pixel which constitutes the whole screen so that it may mention later. 
[0036] And referring to this division field information cc, the above-mentioned division field 
image information extract circuit 38 creates the division image information dd which starts the 
short-time exposure SE from above-mentioned short-time exposure image bb-SE, and it is 
one side, and the division image information ee which starts the long duration exposure LE 
from above-mentioned long duration exposure image bb-LE is created, referring to this 
division field information cc. 

[0037] At this time, as for the division image information ee which as for the division image 
information dd concerning the short-time exposure SE the image information for a background 
is taken out and, on the other hand, starts the long duration exposure LE, the image 
information of a person part is taken out in the example of illustration. 

[0038] Furthermore, before compounding an image for each of such division image information 
dd and ee, gradation amendment is separately carried out by each gradation amendment 
circuits 39 and 40, respectively, the division image information ff and gg after amendment is 
outputted. and the extensive dynamic range (DR) image information hh from which both the 
background and the person are correct exposure in the above-mentioned image composition 
processing circuit 41 is created. 

[0039] Next, drawing 4 is a flow chart which shows the processing in the above-mentioned 
image field dividing network 37. 

[0040] Although the example which generates the division field information cc in this drawing 
4 based on the image information concerning the prolonged exposure LE is explained May 
generate based on the image information concerning the short-time exposure SE, and it does 
not matter even if it generates based on the image information of the both sides of the long 
duration exposure LE and the short-time exposure SE, and Furthermore, what is necessary is 
just to generate the division field information cc or more based on one of image information of 
they, when three or more images generate an extensive dynamic range image. 
[0041] Initiation of this processing reads all the chrominance signals LE of the image 
concerning the long duration exposure LE which was picturized by the above-mentioned 
image sensor 1, and was first inputted through each above-mentioned circuit (x y) (step SI). 
[0042] All these chrominance signals LE (x y) are outputted as signals LEr (x y). LEg (x y), and 
LEb (x y) of each color of r (red), g (green), and b (blue), and x and y show the coordinate 
concerning the array of each pixel which makes the shape of two-dimensional and is arranged, 
and take an integral value here, pointing to the specific location of a coordinate (x y) below — 
integers i and j — using — etc. (i. j) etc. — ** — it indicates. 

[0043] Next, the two-dimensional array Seg (x y) for storing division field information is 
initialized (step 82). 

[0044] And LE image is scanned (step S3). Specifically, the above-mentioned picture signal LE 
(i, j) will be read one by one, incrementing Above i and j. 

[0045] The maximum of each chrominance signals LEr (i, j). LEg (i, j). and LEb (i, j) of the read 
reference pixel LE (i, j) is detected, and the value is stored in Variable CS (step S4). By this 
step S4, the image field dividing network 37 concerned has achieved the function as a 
maximum chrominance-signal detection means. 

[0046] When the value stored in this variable CS judges whether they are one or more 
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predetermined thresholdsTh which show the upper limit of correct exposure (step S5) and 
has not reached a threshold Th 1, the value LEData which shows the information on the 
purport which uses the data of the long duration exposure LE as data of the pixel of a 
coordinate (i. j) is stored in the division field information Seg (i, j) (step S7). 
[0047] moreover, when the values of the above-mentioned variable CS are one or more 
thresholds Th Furthermore, the predetermined threshold Th 2 (it is Th1<Th2 here.) which 
shows the minimum of unsuitable forward exposure It judges whether it is the following (step 
S6), and in being these two or more thresholds Th, it stores in the division field information 
Seg (i, j) the value SEData which shows the information on the purport which uses the data of 
the short-time exposure SE as data of the pixel of a coordinate (i, j) (step S9). 
[0048] On the other hand, the value MIXData which shows the information on the purport 
which uses both the data of the prolonged exposure LE and the data of the short^time 
exposure SE as data of the pixel of a coordinate (ij) is stored in the division field information 
Seg (i, j) noting that it is the middle of correct exposure and unsuitable forward exposure in 
the above-mentioned step S6. when the value of Variable CS has not reached a threshold Th 
2 (step S8). 

[0049] By step S9, the image field dividing network 37 has achieved the function as a 
maximum chrominance-signal comparison means from these steps S5. 
[0050] If it is completed any of the above-mentioned steps S7 and SB and S9 they are, it 
judges whether the scan of the whole screen was completed (step SIO), when having not 
ended, it returns to the above-mentioned step S3, and the following pixel is scanned, and on 
the other hand, when it ends, the division field information Seg (x y) will be outputted (step 
S11), and it will end. In addition, the division field information Seg (x y) outputted becomes the 
signal indicated to be cc in the block diagram ( drawing 2 ). 

[0051] Then, drawing 5 is a flow chart which shows the processing in the above-mentioned 
division field image information extract circuit 38. 

[0052] Initiation of this processing reads the picture signal SE of the total color of the short- 
time exposure SE read from the above-mentioned image data buffer 36 (x y). the picture 
signal LE of the total color of the long duration exposure LE sent through the above- 
mentioned switch 34 (x y), and the above-mentioned division field information Seg (x y) 
outputted from the above-mentioned image field dividing network 37 (step S21). 
[0053] And the fields SegSE (x y) and SegLE (x y) which store division image information are 
initialized (step S22). 

[0054] Next, division field information read at the above-mentioned step S21 is scanned (step 
S23). and it judges first whether the division field information Seg (i, j) is LEData (step S24). 
[0055] Here, in being LEData. it stores the value of LE (i, j) in SegLE (i, j) about each color of 
r, g, and b (step S26). 

[0056] Moreover, in the above-mentioned step S24, it judges whether when it is not LEData, 
the division field information Seg (i, j) is SEData next (step S25), and in being SEData, it 
stores the value of SE (i, j) in SegSE (i, j) about each color of r. g, and b (step S27). 
[0057] Furthermore, in the above-mentioned step S25, while storing the value of LE (i, j) in 
SegLE (i, j) about each color of r, g. and b since it will be Above MIXData in not being SEData, 
the value of SE (i. j) is stored in SegSE (i, j) (step S28). 

[0058] in this way, if it is completed any of the above-mentioned steps S26, S27, and S28 
they are, when judging whether the scan of the whole division field information was completed 
(step S29) and having not ended It returns to the above-mentioned step S23, and the 
following division field information is scanned, and on the other hand, when it ends, the division 
image information SegSE (x y) and SegLE (x y) is outputted (step S30), and it ends. 
[0059] In addition, the division image information SegSE (x y) outputted in the above- 
mentioned step S30 becomes the signal with which the division image information SegLE (x y) 
was indicated to be ee in the block diagram ( drawing 2 ) by the signal indicated to be dd in 
the block diagram ( drawing 2 ), respectively. 

[0060] Next, drawing 6 is the block diagram showing the configuration of the above-mentioned 
gradation amendment circuits 39 and 40. 
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[0061] The Y-signal detector 51 which detects a luminance signal (Ysignal) from the division 
image information dd and ee to which these gradation amendment circuits 39 and 40 are 
outputted from the above-mentioned division field image information extract circuit 38, The 
feature-extraction means slack edge detector 52 which detects whether it is the pixel in 
which this pixel constitutes an edge from a luminance signal mm outputted from this Y-signal 
detector 51 about each pixel in an image. It is based on the edge detecting signal nn and the 
above-mentioned luminance signal mm of this edge detector 52. The histogram generation 
means slack edge histogram count circuit 53 which computes the edge histogram which shows 
the frequency of occurrence to an intensity level about the pixel which constitutes an edge, 
or its near pixel, The gradation amendment property generation means slack tone curve 
setting circuit 54 which computes the tone curve which serves as a gradation amendment 
property based on the accumulation edge histogram signal oo outputted from this edge 
histogram count circuit 53. It is based on the tone curve property signal pp and the above- 
mentioned luminance signal mm which are outputted from this tone curve setting circuit 54, 
and the above-mentioned division field information cc. The above-mentioned division image 
information dd, It has the gradation amendment means slack division field picture signal 
amendment circuit 55 which performs gradation amendment of ee and outputs the division 
image information ff and gg after amendment, and is constituted. 

[0062] In addition, the above-mentioned edge detector 52 outputs the binary information 
which sets edge information to "EG", otherwise, sets edge information to "0" noting that it 
becomes in the circuit which filters by common edge detection operators, such as Sobel, and 
an edge exists in a reference location, if the reinforcement by this edge operator is beyond a 
predetermined threshold (for example, maximum in the range of above-mentioned A/D 
converter 5). 

[0063] In this way, by performing gradation amendment by the circuit as shown in drawing 6 , 
since flattening of a histogram is made, when especially the histogram of a subject-copy 
image is what is concentrated on a specific range, the contrast of an image can be improved. 
Such gradation amendment is the gradation amendment which enables it to perform image 
composition near the center section of the concentration range of the extensive dynamic 
range image which it is going to generate by the image composition processing circuit 41 later. 

[0064] Moreover, in the above-mentioned edge histogram count circuit 53, in case drawing 7 
calculates an accumulation edge histogram, it is drawing showing the example to which weight 
is changed according to the pixel location in an image. 

[0065] First, drawing 7 (A) is an example which performs weighting only in consideration of a 
center section, for example, to counting this pixel as 1 , when an edge exists in a 
circumference part, when an edge exists in a central part, it counts this pixel as 16. 
[0066] As opposed to on the other hand, counting this pixel among 1, when drawing 7 (B) is an 
example which performs gently-sloping weighting toward a periphery focusing on a central 
part, for example, the pixel of a neighboring corner is an edge The number of counts is 
reduced as this pixel is counted as 64, such a high count that it is close to a center section 
about those middle pixels is performed and it goes to a periphery, when the pixel of a central 
part is an edge. 

[0067] In addition, in which example of these drawing 7 (A) and drawing 7 (B), although 
weighting of the exponentiation of 2 is performed in order to make an operation easy, of 
course, it is not limited to this. 

[0068] Here, the example which performs gradation amendment so that image composition 
can be carried out with reference to drawing 8 near the center section of the concentration 
range of the extensive dynamic range image which it is going to generate by the above- 
mentioned image composition processing circuit 41 is explained. 

[0069] Drawing 8 (A) is the histogram of a luminance signal in case the brightness of a 
photographic subject divides and concentrates on comparatively bright brightness and 
comparatively dark brightness. 

[0070] For example, when main photographic subjects exist in the comparatively dark interior 
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of a room with an aperture and the view has spread outside [ comparatively bright ] fine 
weather etc. besides the aperture, the case where he wants to reproduce both an indoor 
photographic subject and the background besides an aperture as an image is considered. 
[0071] At this time, the histogram of the luminance signal in each image comes to be shown, 
for example in this drawing 8 (A), and the part (LE-Image) of a photographic subject inclines 
toward the comparatively darker one, and forms a peak with narrow width of face, and the 
part (SE-Image) of a background inclines toward the comparatively brighter one, and forms 
the peak with narrow width of face similarly. 

[0072] Then, in the gradation amendment circuits 39 and 40 as shown in above-mentioned 
drawing 6 , gradation amendment is processed so that image composition can be carried out 
near the center section (YMAX/2) of the concentration range of the extensive dynamic range 
image which it is going to generate. 

[0073] Drawing 8 (B) is the histogram of the luminance signal when performing gradation 
amendment so that image composition can do the photographic subject of above-mentioned 
drawing 8 (A) near the center section of a concentration range, and the dynamic range of a 
synthetic image can be expanded substantially and can reproduce each image information now 
appropriately from a bright place to a dark place. 

[0074] In this way, much image information can be reproduced in the limited dynamic range, 
and a legible extensive dynamic range image can be generated as a result, using a 
concentration range effectively. 

[0075] Next, drawing 9 is a flow chart which shows the processing in the above-mentioned 
image composition processing circuit 41 . 

[0076] Initiation of this processing first reads the division image information SegSE (x y) and 
SegLE (x y) which is outputted from the above-mentioned gradation amendment circuits 39 
and 40 and by which gradation amendment was carried out. and the division field information 
Seg (x y) outputted from the above-mentioned image field dividing network 37 (step S41). 
[0077] Then, the field WDR (x y) which stores extensive dynamic range image information is 
initialized (step S42), and division field information is scanned (step S43). 
[0078] And it judges whether the division field information Seg (i, j) is LEData (step S44). and 
in being LEData, it stores the data of SegLE (i, j) in the above-mentioned field WDR (i, j) about 
each r, and g and b (step S49). 

[0079] On the other hand, it judges whether when it is not LEData in the above-mentioned 
step S44 next, the division field information Seg (i. J) is SEData (step S45). and in being 
SEData, it stores the data of SegSE (i. j) in the above-mentioned field WDR (i. j) about each r. 
and g and b (step 850). 

[0080] Moreover, in the above-mentioned step S45, in not being SEData, the maximum CS of 
each color component of SegLE (i, j), SegLEr. SegLEg, and the SegLEb(s) is detected (step 
S46), and next a random number R is generated, and it acquires, and sets the selection 
characteristic value ST as 0 or 1 according to the magnitude of the random numbers R and 
CS (step S47). 

[0081] As the setting approach of this selection characteristic value ST, the function Cst 
(CS) which shows what magnitude CS is is defined here like a formula 1 about the 
chrominance-signal maximum CS in the range (the range of the threshold Th 1 in drawing 4 . 
and a threshold Th 2) in which division field information can serve as MIXData, for example. 
[0082] 
[Equation 1] 



[0083] This function Cst (CS) can change the rate which chooses division image information 
(SegLE (i, j), SegSE (i, j)) according to the magnitude of CS, in order to carry out the 
increment in monotone from 0 into a domain (Thi <=CS<Th2) 1. 

[0084] Next, it can judge at random which division image information is chosen in the place 
which acquired the random number R by multiplying the above-mentioned function Cst (CS) 



Cst(CS) - 



(7»1S3<I»2) 
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by the maximum RMAX of a random number R, taking the magnitude of CS into consideration. 

This decision determines the selection characteristic value ST. In this way, the selection 

characteristic value ST is expressed like a formula 2. 

[0085] 

[Equation 2] 

fO :Cst(CS)xRMAX^R 



I 1 tOthermze 

[0086] In this formula 2, the selection characteristic value ST is set up in the form depending 
on the magnitude of the chrominance-signal maximum CS. That is. Function Cst (CS) is small 
(the degree of a white jump is comparatively weak about the division image information SegLE 
(i, j) currently referred to). When the property of a long duration exposure LE image remains, 
the inclination for ST to be set to 1 becomes strong. The inclination, as for ST, for Function 
Cst (CS) to be set to 0 on the contrary in being large (the degree of a white jump is 
comparatively strong and the property of a long duration exposure LE image does not remain 
about the division image information SegLE (i, j) currently referred to) becomes strong. 
[0087] And it judges whether the selection characteristic value ST is 1 (step S48), in being 1, 
it goes to the above-mentioned step S49, the data of SegLE (i, j) are stored in the above- 
mentioned field WDR (i, j) about each r, and g and b, on the other hand, in being 0, it goes to 
the above-mentioned step S50, and the data of SegSE (i, j) are stored in the above- 
mentioned field WDR (i, j). 

[0088] By processing of step S44 to the step S50 which was mentioned above, the image 
composition processing circuit 41 has achieved the function as an image composition means. 
[0089] If the above-mentioned step S49 or step S50 is completed, it judges whether the scan 
of the whole division field information was completed (step S51), when having not ended, it 
returns to the above-mentioned step S43. and the following division field information is 
scanned, and on the other hand, when it ends, the extensive dynamic range image information 
WDR (x y) will be outputted (step S52), and it will end. 

[0090] Thus, by choosing division image information in consideration of the magnitude of 
chrominance-signal maximum, the field whose division field information is MIXData has the 
effectiveness similarly displayed as the condition of having performed dithering, and can 
control the sense of incongruity to the prolonged exposure LE and the short-time exposure 



[0091] In addition, although **** explained the case where an extensive dynamic range image 
was obtained using the image group which becomes by two images, a prolonged exposure 
image and a short-time exposure image, even if it uses the image of exposure conditions with 
which a large number differ more, it cannot be overemphasized that an extensive dynamic 
range image can be obtained similarly. 

[0092] Moreover, although the example which applied the image processing system to the 
electronic camera here, and the example applied to the camera digital disposal circuit of an 
electronic camera in more detail were explained, of course, it cannot restrict to this and can 
apply to various kinds of equipments which perform an image processing widely. 
[0093] In order according to such 1st operation gestalt to divide into a proper exposure field 
and an unsuitable forward exposure field based on a picture signal and to perform gradation 
amendment only about a proper exposure field about each image of the image group picturized 
on different exposure conditions to the same photographic subject, the contrast of a proper 
exposure field is improvable. 

[0094] Moreover, by compounding the proper exposure field of each image after gradation 
amendment, where contrast is held over a large area from a dark part to a bright part, an 
extensive dynamic range image can be obtained. 

[0095] Furthermore, since gradation amendment is performed so that image composition can 
be performed near the center section of the concentration range of the extensive dynamic 
range image which it is going to generate, it becomes possible to use a concentration range 
effectively and to consider as a legible image. 




SE. 
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[0096] And in a gradation amendment circuit, the histogram in consideration of the description 
of an image is generated by generating a histogram from the near pixel of the edge component 
extracted by filtering. 

[0097] Moreover, gradation amendment in consideration of the description of an image can be 
performed by generating a gradation amendment property based on this histogram, and using 
this for gradation amendment of a picture signal. Therefore, the contrast about the near pixel 
of the description component is improvable. Furthermore, the configuration of the whole 
processing can be simplified by using easy processing called a histogram. 
[0098] In addition, since it was made to change weighting at the time of generating a 
histogram according to the pixel location in an image, it becomes possible to depend in 
consideration of the location of main photographic subjects etc.. and to perform suitable 
gradation amendment. 

[0099] Moreover, in an image field dividing network, by detecting the maximum of two or more 
chrominance signals which constitute a picture signal about each image in an image group, and 
comparing the maximum of this chrominance signal with predetermined picture signal level, 
judgment of a proper exposure field (a middle exposure field is included) and an unsuitable 
forward exposure field can be performed, and the level of all chrominance signals can make a 
proper field a proper exposure field. Thereby, the precision of the proper exposure field 
divided by the image field dividing network can be improved. 

[0100] Drawing 15 shows the 2nd operation gestalt of this invention from drawing 10 . In this 
2nd operation gestalt, the sign same about the same part as the 1 st above-mentioned 
operation gestalt is attached, explanation is omitted, and only a mainly different point is 
explained. 

[0101] The configuration of the electronic camera of this 2nd operation gestalt is the same as 
that of what was shown in drawing 1 of the 1st operation gestalt mentioned above almost. 
[0102] Next, drawing 10 is the block diagram showing the configuration of a camera digital 
disposal circuit. 

[0103] In the camera digital disposal circuit of this operation gestalt, the picture signal which 
the picture signal from the above-mentioned switch 34 requires for the short-time exposure 
SE is inputted into the Y/C separation circuit 61, and the picture signal concerning one long 
duration exposure LE is inputted into the Y/C separation circuit 62. 

[0104] In these Y/C separation circuits 61 and 62, based on r of an input picture signal, g, and 
b component (R, G, B). as shown in a formula 3 at a luminance signal Y (refer to drawing 12 
(A)) and color-difference signals Cb and Cr, it dissociates. 
[0105] 

[Equation 3] Y = 0.29900R+0.58700G+0.14400BCb=-0.16874R-0.33126G+0.50000BCr= 
0.50000R-0.41869G-0.08131B [0106] In this way. in the feature-extraction means slack edge 
detector 63 whose luminance-signal mm-SE concerning the short-time exposure SE of the 
separated inside is a part of image amendment means, luminance-signal mm-LE concerning 
the prolonged exposure LE is inputted into the said feature-extraction means slack edge 
detector 64, respectively, and an edge component is extracted by well-known secondary 
differentiation filters, such as Laplacian, (refer to drawing 12 (B)). In addition, since the 
secondary differentiation filter is used, a forward detection result and a negative detection 
result (refer to drawing 12 mentioned later (O) will be outputted here. 

[0107] In this way, based on edge information signal nn-SE and nn-LE which were detected, 
and luminance-signal mm-SE from the above-mentioned Y/C separation circuits 61 and 62. 
mm-LE and color-difference-signal qq-SE and qq-LE, gradation amendment is performed to 
the image concerning the short-time exposure SE, and the image concerning the long duration 
exposure LE in the image amendment means slack gradation amendment circuits 65 and 66, 
respectively. 

[0108] Drawing 1 1 is the block diagram showing the configuration of these gradation 
amendment circuits 65 and 66. In addition, in this drawing 1 1 , in order to simplify explanation, 
explanation about the gradation amendment circuit 66 which mainly starts the prolonged 
exposure LE is given. 
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[0109] In the gradation amendment circuit 66. edge information signal nn-LE based on a 
luminance signal is inputted into the description component histogram generation means slack 
edge histogram creation section 71. and the edge histogram which shows the frequency of 
occurrence to the level of edge information signal nn-LE is created. In addition, unlike the 
edge histogram in the operation gestalt of the above 1 st, the edge histogram created here is a 
histogram of the edge component of a luminance signal itself. 

[01 10] In a common natural image, an edge histogram as it known that the above-mentioned 
edge histogram can be modeled by Gaussian distribution, for example, shown in drawing 12 (C) 
is created. 

[01 1 1] In this way, two thresholds of **2sigma are set up by inputting into the threshold 
calculation means slack threshold calculation section 72 the edge histogram created in the 
edge histogram creation section 71, and computing the standard deviation sigma of an edge 
histogram there, for example, making it twice this standard deviation sigma into a threshold. 
[0112] Based on edge information signal nn-LE of the threshold outputted from this threshold 
calculation section 72, and the above-mentioned luminance signal, and luminance-signal mm- 
LE, the object pixel selection means slack luminance-signal selection section 73 extracts the 
luminance signal of the pixel which corresponds to a larger edge component than above- 
mentioned threshold +2sigma or an edge component smaller than threshold-2sigma among the 
edge components of a luminance signal. 

[01 13] Thus, only a stably effective edge component can be extracted by defining a threshold 
based on the standard deviation sigma of an edge histogram, without being dependent on 
exposure conditions. 

[01 14] In this way, based on the luminance signal of the extracted effective edge component, 
a brightness histogram as shown, for example in drawing 12 (D) is created in the histogram 
generation means slack brightness histogram creation section 74. 

[01 1 5] In order to generate a high contrast image in the distribution modeling means slack 
target histogram creation section 75 based on the brightness histogram created in this 
brightness histogram creation section 74, processing which changes the above-mentioned 
brightness histogram in the shape of Gaussian one is performed. 

[01 16] that is, the thing changed in the shape of Gaussian one with this operation gestalt 
although the approach of generating a high contrast image by carrying out flattening of the 
histogram in the 1 st operation gestalt mentioned above was used — more — high — a 
contrast image is obtained. 

[01 17] However, since the shape of what Gaussian one of a histogram is set up differ for 

every image, it is necessary to perform the optimal setup accommodative. 

[01 18] Here, a means to obtain the histogram used as a target in application is used by 

carrying out the convolution of the brightness histogram using a Gaussian kernel. 

[0119] That is. when the histogram which changed the brightness histogram into O (L) and 

changed this in the shape of Gaussian one is set to T (L), the operation for changing into T (L) 

from O (L) comes to be shown in a formula 4. 

[0120] 

[Equation 4] ^ 

[0121] A notation expresses a convolution and L means the level of a luminance signal 
here. Moreover, k becomes the parameter for adjustment of to what extent to change in the 
shape of Gaussian one. and uses about 1 to 2 times of the standard deviation of brightness 
histogram O (L) in this operation gestalt, for example. 

[0122] In this way, in this example, a target histogram as shown in drawing 1 2 (E) is created. 
[0123] Thus, in order to generate the histogram which is inputted and which serves as a 
target for every image in application, it can respond flexibly also to various images. 
[0124] In this way. in the gradation amendment property generation means slack conversion 
curvilinear creation section 76. the obtained target histogram and the above-mentioned 
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brightness histogram generate the gray-scale-conversion curve whicn changes a brightness 
histogram into a target histogram, as shown in drawing 1 2 (F), and based on this gray-scale- 
conversion curve, a luminance signal is changed by the gradation amendment means slack 
luminance-signal transducer 77. 

[0125] And based on the luminance signal and color-difference-signal qq-LE after conversion 
from the above-mentioned luminance-signal transducer 77, as the brightness color difference 
composition section 79 is shown in the following formula 5, it changes into an RGB code and 
outputs to the image composition means slack image composition processing circuit 67. 
[0126] 

[Equation 5] R = Y+1.40200CrG = Y-0.34414Cb-0.7141 7GrB = Y+1.77200Cb[0127] On the 
other hand, detection of the maximum (LE-YMAX) of the luminance signal after conversion 
from the above-mentioned luminance-signal transducer 77 and the minimum value (LE-YMIN) 
is performed in the maximum minimum value detecting element 78. and it is as a result of 
[ the ] detection (it becomes rr-LE in drawing 10 .). Moreover, it is rr-SE [ in / in the case of 
short-time exposure / drawing 10 ]. It outputs to the above-mentioned image composition 
processing circuit 67. 

[0128] Next, drawing 1 3 and drawing 14 are flow charts which show the processing in the 
above-mentioned image composition processing circuit 67. In addition, these drawing 13 and 
drawing 14 divide a series of actuation by the image composition processing circuit 67 into 
two drawings on account of a publication. 

[0129] Initiation of this processing reads the maximum of the proper exposure field brightness 
which consists of a luminance signal by which gray scale conversion was first chosen and 
carried out in each image information SE (x y) and LE (x y) and the above-mentioned 
luminance-signal selection section 73 which are outputted from the above-mentioned 
gradation amendment circuits 65 and 66, and by which gradation amendment was carried out 
and minimum value information (SE-YMAX, SE-YMIN), and (LE-YMAX, LE-YMIN) (step S61). 
[0130] Then, the field SA (x y) which stores the smoothing field information for distinguishing 
whether data smoothing is performed in Field WDR (x y) and the latter part which store 
extensive dynamic range image information is initialized (step S62), and LE image information 
is scanned (step S63). 

[0131] And based on LEr (i, j), LEg (i, j). and LEb (i, j) concerning r, g, and b which were read, 
as shown in a formula 6, a luminance signal is created (step S64). 

[Equation 6] LEy=0.3LEr(i, j)+0.59LEg(i, j)+0.11 LEbO, j) [0132] And when this LEy is below LE- 
YMAX, and it judges whether it is smaller than SE-YMIN (step S65) and both conditions are 
fulfilled, the data of LE (i, j) are stored in the above-mentioned field WDR (i, j) about each r, 
and g and b (step S67). 

[0133] On the other hand, when [ when one / at least / conditions are not fulfilled in the 
above-mentioned step S65 next] LEy is larger than LE-YMAX, and judges whether it is more 
than SE-YMIN (step S66) and both conditions are fulfilled, the data of SE (i, j) are stored in 
the above-mentioned field WDR (i, j) about each r, and g and b (step S68). 
[0134] In step S68, the image composition processing circuit 67 has achieved the function as 
a proper exposure pixel selection means from these steps S65. 

[0135] moreover, in the above-mentioned step S66, when one [ at least ] conditions are not 
fulfilled 1 is stored in the smoothing field information SA (i, j) that it judges whether low pass 
filter processing mentioned later is performed while storing the average of the data of LE (i, j), 
and the data of SE (i, j) in the above-mentioned field WDR (i. j) about each r, and g and b (step 
S69). In this step S69, the image composition processing circuit 67 has achieved the function 
as an averaging means. 

[0136] And it judges whether the scan of the whole screen was completed (step S70), when 
having not ended, it returns to the above-mentioned step 863, and the following pixel is 
scanned, and on the other hand, when it ends, the smoothing field information about each 
pixel is scanned (step S71). 

[0137] And it judges whether the smoothing field information SA (i, j) is 1 (step S72), and in 
being 1 , it performs low pass filter processing according to a luminance signal and a color- 
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difference signal about Sx^pixels centering on WDR (i, j) (step S73).Th this step S73. this 
image composition processing circuit 67 has achieved the function as a smoothing means. 
[0138] On the other hand, when the smoothing field information SA (i, j) is judged not to be 1 
in the above-mentioned step S72, actuation of this step S73 is skipped. 
[0139] And when judging whether the scan about all pixels was completed (step S74). 
returning to the above-mentioned step S71. scanning again, when having not ended, and 
having ended, the extensive dynamic range image information WDR (x y) is outputted (step 
S75), and it ends. 

[0140] As by performing such processing shows to drawing 1 5 The field where the pixel 
brightness value was inserted into SE-YMIN and LE-YMAX (like drawing 1 5 (A)) Information 
which uses the average of the data of the prolonged exposure LE and the data of the short- 
time exposure SE about a brightness region while belonging to both proper exposure fields and 
belonging to neither of the proper exposure fields like a brightness region or drawing 15 (B) 
will be mixed (Mixture). Any of SE-YMIN and LE-YMAX or the image (LE-image) of the long 
duration exposure about a pixel darker than the one where brightness is lower is used. The 
image (SE-image) of short-time exposure will be used about a pixel brighter than the one 
where any of SE-YMIN and LE-YMAX or brightness is higher. 

[0141] In this case, since the image (LEHmage) of long duration exposure will be used about a 
pixel darker than LE-YMIN and the image (SEHmage) of short-time exposure will be used 
about a pixel brighter than SE-YMAX, when there is no pixel in proper exposure within the 
limits, a near thing will be chosen more appropriately. 

[0142] In this way, in case two or more proper exposure fields are compounded and the 
extensive dynamic range image of 1 is generated When the lack field which turns into an 
unsuitable forward exposure field in each images of all in the image group which becomes by 
the long duration exposure image and the short-time exposure image exists When this lack 
field is exposure over, using the applicable field of the minimum exposure image in the above- 
mentioned image group, when this lack field is an exposure undershirt on the other hand, the 
applicable field of the maximum exposure image in the above-mentioned image group will be 
used, and the role of a field adjustment means to fill up the above-mentioned lack field is 
played. 

[0143] It is the same as that of the 1st operation gestalt mentioned above that return to 
drawing 10 again, and the data of the compounded extensive dynamic range image are 
outputted through a switch 33 after synthetic processing which was mentioned above in the 
image composition processing circuit 67 is performed. 

[0144] While doing so the almost same effectiveness as the 1st operation gestalt mentioned 
above according to such 2nd operation gestalt, the contrast of each image is improvable by 
performing gradation amendment about each image of the image group picturized on different 
exposure conditions to the same photographic subject. 

[0145] Moreover, by compounding each image by which gradation amendment was carried out, 
in order to generate an extensive dynamic range image, where contrast is held over a large 
area from a dark part to a bright part, an extensive dynamic range image can be obtained. 
[0146] Furthermore, in a gradation amendment circuit, the description which is proper 
exposure can choose a remarkable pixel from distribution of the description component 
histogram by generating the histogram about the edge component by filtering, and choosing 
from the distribution condition the object pixel used for generation of a gradation amendment 
property. 

[0147] And in each image, the contrast of an important part is improvable by making 
conversion for generating the histogram of an object pixel and modeling this in predetermined 
distribution into a gradation amendment property. 

[0148] In addition, in an image composition processing circuit, when two or more picture 
signals by which gradation amendment was carried out about the same pixel exist, each 
picture signal can be equally dealt with by taking the averaging of each picture signal. 
Furthermore, the sense of incongruity produced between the picture signals which have not 
carried out averaging can be controlled by graduating to the pixel by which averaging was 
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carried out to the minimum. 
[0149] Moreover, in an image composition processing circuit, by investigating each image 
which performed gradation amendment for every pixel location, and choosing the pixel near 
the proper exposure in a pixel group, the signal of unsuitable forward exposure can be 
eliminated and only the signal more near proper exposure can be used for an extensive 
dynamic range image. 

[0150] furthermore, in case a proper exposure field is compounded and an extensive dynamic 
range image is generated in an image composition processing circuit, when a lack field exists 
By filling up a lack field using the applicable field of the maximum exposure image in an image 
group, when a lack field is exposure over and a lack field is an exposure undershirt about the 
applicable field of the minimum exposure image in an image group Since a lack field can be 
amended for the optimal information in an image group, the sense of incongruity in a lack field 
can be controlled to the minimum. 

[0151] In addition, as for this invention, it is needless to say for various deformation and 
application to be possible within limits which are not limited to the operation gestalt 
mentioned above and do not deviate from the main point of invention. 
[0152] 

[Effect of the Invention] In order according to the image processing system of this invention 
by claim 1 to compound these and to generate the extensive dynamic range image of 1 after 
performing gradation amendment for every image as explained above, the extensive dynamic 
range image with which contrast was held from the low brightness region to the high 
brightness region can be obtained. 

[0153] Moreover, in order according to the image processing system of this invention by claim 
2 to compound these and to generate the extensive dynamic range image of 1 after 
performing gradation amendment of the divided proper exposure field for every image, the 
extensive dynamic range image with which contrast was held from the low brightness region 
to the high brightness region can be obtained. 

[0154] Furthermore, in order according to the image processing system of this invention by 
claim 3 to perform gradation amendment so that image composition can be performed near 
the center section of the concentration range of the extensive dynamic range image which it 
is going to generate while doing so the same effectiveness as invention according to claim 1 
or 2. it becomes possible to use a concentration range effectively and to consider as a legible 
image. 

[0155] According to the image processing system of this invention by claim 4, while doing so 
the same effectiveness as invention according to claim 1 or 2, in an image amendment means, 
the histogram in consideration of the description of an image is generable by generating a 
histogram from the near pixel of the description component by filtering. Moreover, gradation 
amendment in consideration of the description of an image can be performed by generating a 
gradation amendment property based on a histogram, and using this for gradation amendment 
of a picture signal. Therefore, it becomes possible to improve the contrast about the near 
pixel of the description component. Furthermore, the configuration of the whole processing 
can be simplified by using easy processing called a histogram. 

[0156] Since it was made to change weighting at the time of generating a histogram according 
to the pixel location in an image according to the image processing system of this invention by 
claim 5 while doing so the same effectiveness as invention according to claim 4, it becomes 
possible to depend in consideration of the location of main photographic subjects etc., and to 
perform suitable gradation amendment. 

[0157] While doing so the same effectiveness as invention according to claim 1 or 2 according 
to the image processing system of this invention by claim 6 By generating the histogram 
about the description component by filtering, and choosing the object pixel used for 
generation of a gradation amendment property from the distribution condition in an image 
amendment means When a proper exposure field exists, the description can choose a 
remarkable pixel in it, and on the other hand, when a proper exposure field does not exist, the 
description which is proper exposure can choose a remarkable pixel from distribution of the 
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description component histogram. Moreover, it becomes possible by making conversion for 
generating the histogram of an object pixel and modeling this in predetermined distribution 
into a gradation amendment property to improve the contrast of an important part in each 
image. Furthermore, the configuration of the whole processing can be simplified by using easy 
processing called a histogram. 

[0158] While doing so the same effectiveness as invention according to claim 1 or 2, when two 
or more picture signals by which gradation amendment was carried out about the same pixel 
location exist in an image composition means according to the image processing system of 
this invention by claim 7, each picture signal can be equally dealt with by taking the averaging 
of each picture signal. Moreover, the sense of incongruity produced between the picture 
signals which have not carried out averaging can be controlled by graduating to the pixel by 
which averaging was carried out to the minimum. 

[0159] According to the image processing system of this invention by claim 8, while doing so 
the same effectiveness as invention according to claim 1, in an image composition means, by 
investigating each image which performed gradation amendment for every pixel location, and 
choosing the pixel near the proper exposure in a pixel group, the signal of unsuitable forward 
exposure can be eliminated and only the signal more near proper exposure can be used for an 
extensive dynamic range image. 

[0160] While doing so the same effectiveness as invention according to claim 1 or 2 according 
to the image processing system of this invention by claim 9 In case a proper exposure field is 
compounded and an extensive dynamic range image is generated in an image composition 
means, when a lack field exists By filling up a lack field using the applicable field of the 
maximum exposure image in an image group, when a lack field is exposure over and a lack field 
is an exposure undershirt about the applicable field of the minimum exposure image in an 
image group Since a lack field can be amended for the optimal information in an image group, 
the sense of incongruity in a lack field can be controlled to the minimum. 
[0161] While doing so the same effectiveness as invention according to claim 2 according to 
the image processing system of this invention by claim 10 By detecting the maximum of two 
or more chrominance signals with the maximum chrominance-signal detection means about 
each image in an image group, and comparing the maximum of this chrominance signal with 
predetermined picture signal level in a division means Judgment of a proper exposure field and 
an unsuitable forward exposure field can be performed, and the level of all chrominance 
signals can make a proper field a proper exposure field. It becomes possible to improve by this 
the precision of the proper exposure field divided by the division means. 
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1This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 

[Brief Description of the Drawings] 

[Drawing 1] The block diagram showing the fundamental configuration of the electronic 
camera of the 1st operation gestalt of this invention. 

[ Drawin g 2] The block diagram showing the configuration of the camera digital disposal circuit 
in the operation gestalt of the above 1st. 

[Drawin g 3] Drawing showing the situation of the processing signal in the camera digital 
disposal circuit of the operation gestalt of the above 1 st. 

[ Drawin g 4] The flow chart which shows the processing in the image field dividing network of 
the operation gestalt of the above 1 st. 

[ Drawing 5] The flow chart which shows the processing in the division field image information 
extract circuit of the operation gestalt of the above 1 st. 

[Drawing 6] The block diagram showing the configuration of the gradation amendment circuit 
in the operation gestalt of the above 1 st. 

[Drawing 7] Drawing showing the example to which weight is changed according to the pixel 
location in an image in the edge histogram count circuit of the operation gestalt of the above 
1st in case an accumulation edge histogram is calculated. 

[ Drawin g 8] The histogram of a luminance signal in case the brightness of a photographic 
subject divides and concentrates on comparatively bright brightness and comparatively dark 
brightness in the operation gestalt of the above 1st, and the histogram of the luminance signal 
when performing gradation amendment so that image composition can do the photographic 
subject near the center section of a concentration range. 

[ Drawin g 9] The flow chart which shows the processing in the image composition processing 
circuit of the operation gestalt of the above 1st. 

[Drawin g 10] The block diagram showing the configuration of the camera digital disposal 
circuit in the 2nd operation gestalt of this invention. 

[ Drawing 1 1] The block diagram showing the configuration of the gradation amendment circuit 
in the operation gestalt of the above 2nd. 

[Drawing 12 ] Drawing showing the graph in the process when creating a brightness histogram, 
a target histogram, and a conversion curve from the edge of a luminance signal etc. in the 
operation gestalt of the above 2nd. 

[ Drawing 1 3] The flow chart which shows a part of processing in the image composition 
processing circuit of the operation gestalt of the above 2nd. 

[ Drawing 14 ] The flow chart which shows a part of other processings in the image composition 
processing circuit of the operation gestalt of the above 2nd. 

[Drawin g 1 5 ] Drawing showing the situation of the image synthetic model in the case of 
belonging to the case where an inside brightness region belongs to both proper exposure 
fields, and neither of the proper exposure fields, in the operation gestalt of the above 2nd. 
[ Drawing 16] The diagram showing the situation of the processing which creates the 
conventional extensive dynamic range picture signal, and compresses it. 
[Description of Notations] 

6 — Camera digital disposal circuit (image processing system) 

37 — Image field dividing network (a division means, the maximum chrominance-signal 
detection means, the maximum chrominance-signal comparison means) 
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38 — Division field image information extract circuit (division means/ 

39, 40, 65, 66 — Gradation amendment circuit (image amendment means) 

41 67 — Image composition processing circuit (an image composition means, an averaging 

means, a smoothing means, a proper exposure pixel selection means, field adjustment means) 

51 — Y-signal detector 

52, 63, 64 — Edge detector (feature-extraction means) 

53 — Edge histogram count circuit (histogram generation means) 

54 — Tone curve setting circuit (gradation amendment property generation means) 

55 — Division field picture signal amendment circuit (gradation amendment means) 
61 62 — Y/C separation circuit 

71 — Edge histogram creation section (description component histogram generation means) 

72 — Threshold calculation section (threshold calculation means) 

73 — Luminance-signal selection section (object pixel selection means) 

74 — Brightness histogram creation section (histogram generation means) 

75 — Target histogram creation section (distribution modeling means) 

76 — Conversion curvilinear creation section (gradation amendment property generation 
means) 

77 — Luminance-signal transducer (gradation amendment means) 

78 — The maximum minimum value detecting element 

79 — Brightness color difference composition section 



[Translation done.] 
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